Epsilon Equitable Partition: On Scheduling Data Loading and View Maintenance in Soft Real-time Data Warehouses

Data warehouses contain historic data providing information for analytical processing, decision making and data mining tools. However, several business intelligence applications nowadays require access to real-time data to make sound decisions. As a consequence, there is a great demand to incorporate new data from sources to the data warehouse as fast as possible. That motivates the construction of real-time data warehouse. Despite high demand, moving from a traditional data warehouse to a real-time one is not straightforward since we have to deal with the problem of efficiently scheduling various activities, viz., updates, view maintenances and OLAP transactions in a timely manner. In addition, OLAP transactions are now associated with deadlines (transaction timeliness) and data freshness requirement (data timeliness). Balancing between these two requirements poses another challenge in real-time data warehousing context. In this paper, we present an efficient technique aimed at updating data and performing view maintenance for real-time data warehouses while still enforcing these two timing requirements for the OLAP transactions. Our proposed approach aims at addressing the issues of applying updates and performing view maintenance while still effectively serving user OLAP queries in real-time data warehouse environment. Through extensive empirical studies, we demonstrate the efficiency of ORAD in achieving the goal of building a real-time data warehouse.